Facetop: An Integrated Desktop/Video Interface for Individual Users and Paired Collaborations

Use of video in distributed pair programming was discussed in a workshop at the XP/Agile Universe 2002 conference. This workshop was attended by over 30 people, most of whom had tried some form of distributed pair programming and were working on tools to improve the effectiveness of such activities. The consensus on video was that “webcam” style, postage stamp video – small image and low frame rate – was of little value in enhancing communications or sense of presence in a distributed pairing. However, it was felt that video, if large enough and real enough, was of potential value and worth further research. We have been doing that research since that time. The Transparent Video Facetop is a novel integration of the traditional PC desktop with a live video signal of the PC user. Using an inexpensive Firewire video camera located atop the monitor and pointed back at the user, Facetop captures a 640 x 480 pixel stream at 30 frames per second. The video image is then made full-screen, mirrored, made semi-transparent, and finally composited into the desktop image in the system frame buffer. The result is the appearance of the user “behind” the desktop, and able to point at items on the screen by raising a hand into the camera’s field of view. One feature of Facetop is the ability to track the user’s finger, allowing mouse control by hand. We have also built a two-person version where two Facetops on two different computers share their video images over the network and composit both onto a shared desktop, facilitating synchronous pair collaborations. In this paper we explain the system architecture that allows successful implementation and acceptable network performance; we describe our initial usability trials; and we discuss the use of Facetop in the application domains of pair programming and universal access for people with audio impairments. COTS Tools: Effective but lacking In our previous dPP experiments, programmers worked as a pair using COTS software, including pcAnywhere (Symantec) and Yahoo messenger. The pcAnywhere application provided a shared desktop, so that the two separated programmers are effectively working on a single host computer; each saw exactly the same desktop as they would have seen sitting side-by-side at the host PC. Yahoo messenger provided voice communications, and occasional text exchange. There was no video information. Our experiments, involving approximately 50 teams over 3 different trials, found that people working as synchronous pairs in this distributed environment were as effective and efficient at programming as co-located pairs were. In post-experimental interviews, teams consistently told us 3 things: 1. The Problem, Background, and Approach Our research into video-based interfaces was motivated by several human communication problems that arose in our studies of a paired, synchronous collaborative activity called distributed pair programming (dPP). Pair programming is a software engineering technique where two programmers sit at one PC to develop code; it is an important practice in several recent agile development methods, the most well known being Extreme Programming (XP) [1,2]. One programmer types (“drives”) while the other reviews and assists (“navigates”); roles swap frequently. • They missed facial expressions and the sense of presence that one has with a co-located partner; • They wanted a way to point at the shared work they were discussing via the audio channel, as they would do with fingers when co-located; • They wanted a whiteboard for drawing and design work. The benefits of pair programming are well known for co-located participants [3,17,19]; these studies have shown that synchronously collaborating, co-located pairs produce better code (fewer errors) in less time than teams of individuals. For the past 3 years we have been studying distributed pairs, where the programmers are physically separated and communicating over the Internet [6,7,15]. The questions under study were (a) can it be done, and (b) do the benefits remain in distributed contexts? Our results indicate “yes” and “yes” so far. The Facetop provides potential solutions to each of these problems via its video capabilities. The video capabilities in Facetop are very different from the “webcam” style video found in shared desktop collaboration software like Microsoft Shared Desktop or NetMeeting. The image is large, and frame rates run from 15 to 30 fps, showing facial details and fine motor movements of the fingers and lips. The video image is also tightly and seamlessly integrated with the shared workspace via transparency, thereby eliminating the “dual” nature of video teleconferencing solutions. Users do not have to switch their attention from desktop, to video, back to desktop. It has been established that distance matters [18]; face-to-face pair programmers will most likely outperform distributed pair programmers in terms of sheer productivity. However, the inevitability of distributed work in industry and education calls for research in determining how to make this type of work most effective. Figure 1 gives two views of the single-user Facetop, showing the camera facing the user, and the semi-transparent, full-screen video image of the user composited with the desktop. The user sees